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DETAILED ACTION 

Response to Arguments 

1 . Applicant's arguments with respect to claims 1 and 28 have been considered but 
are moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 1,8-11,15, 20, 23-24, 26-29, 31 -33, and 35-38 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Monroe (US 6970183 B1) in view of Hsu 
(US 6741744 B1). 

Re claim 1, Monroe discloses a method for the monitoring of an environment by 
use of one or more pairs of imagers, each imager monitoring said environment from a 
different angle of view, comprising the steps of: a) providing one or more pairs of 
imagers (Monroe: Fig. 7); b) positioning both imagers of said one or more pairs of 
imagers along a common vertical line such that they both capture a same object at a 
different angle of view (Monroe: column 17, line 62-column 2, each location has its own 
system node with a plurality of cameras); e) carrying out an unmanned, real-time 
vertical observation of said controlled space or sections thereof, according to the 
observation parameters (Monroe: column 29, lines 36-44); f) determining the distance to 
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said observed objects from said one or more pairs of imagers (Monroe: column 28, lines 
49-52, laser range finder); h) classifying a type and degree of danger of each of said 
observed objects by jointly processing said real-time data obtained from said 
stereoscopic observation with respect to the angle of view of each of said imagers, the 
path and size of, and distance from, said observed objects, and said stored danger 
parameters (Monroe: column 9, lines 37-39 and column 30, lines 38-41 , types of events 
are identified and classified so as to alert particular parties); and i) providing an 
indication when one or more of said observed objects is approaching said controlled 
space and has been classified as having a sufficiently high degree of danger so as to be 
liable of damaging an authorized body within said controlled space (Monroe: column 30, 
lines 7-32). 

Monroe does not explicitly disclose that at least one of said pairs comprises an 
optical imager and at least one of said pairs comprises a Forward Looking Infra Red 
(FLIR) imager; performing a stereoscopic observation of objects; c) defining and storing 
in a memory, programs for processing, in real-time, photographic data to be obtained 
from the stereoscopic observation of objects by use of said one or more pairs of 
imagers; d) determining and storing parameters according to which the observation of a 
controlled space or sections thereof is effected; and g) evaluating the size of each of 
said observed objects. However, Hsu discloses a system including compilable 
language for extracting objects from an image using a primitive image map, wherein a 
computer program is executed to extract objects from an image (Hsu: column 10, lines 
5-18), wherein the algorithm utilizes stereoscopic characteristics (Hsu: column 20, lines 



Application/Control Number: 1 0/521 ,207 Page 4 

Art Unit: 2621 

17-23) and image and map data in FLIR applications (Hsu: column 3, lines 38-60) along 
with object size and other attributes (Hsu: Table I). Since both Monroe and Hsu relate 
to monitoring systems, one of ordinary skill in the art at the time of the invention would 
have found it obvious to combine the principles of Hsu with the system of Monroe in 
order to eliminate the need for tedious pixel level calculations for object extraction (Hsu: 
column 10, lines 47-50). 

Re claim 8, Monroe discloses that the indication is an alarm signaling the 
presence and nature of any dangerous objects, the danger of collisions and possible 
desirable preventive actions (Monroe: column 18, lines 58-61, additional information 
including the type of event may be provided). 

Re claim 9, Monroe discloses that the stereoscopic observation is carried out by 
performing the steps of: a) modifying the angle of one or more imagers (Monroe: 
column 28, lines 52-57, object tracking may include panning the camera); b) capturing 
one or more images with said one or more imagers (Monroe: column 4, lines 1-5, high 
resolution images and video clips are processed by the system); c) processing said 
captured one or more images by a computerized system (Monroe: column 4, lines 6-8, 
processors are used for automatic event assessment); and d) repeating steps a) to c) 
(Monroe: column 37, line 62-column 38, line 5, the cycle repeats). 

Re claim 10, Monroe discloses that the stereoscopic observation is carried out 
as a continuous scan or segmental scan (Monroe: column 7, lines 50-56, the monitoring 
station can monitor the data continuously). 
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Re claim 11, Monroe discloses a) setting initial definition for the stereoscopic 
observation and for the processing of the data of said stereoscopic observation 
(Monroe: column 33, lines 12-23, the system is responsive to programmed events); b) 
storing in the memory the data that represent the last captured one or more images at a 
specific angle of the imagers (Monroe: column 33, lines 12-23, the system learns that 
an object belongs or does not belong at a specific location); and c) processing said data 
for detecting suspected objects, by performing, firstly, pixel processing and secondly, 
logical processing (Monroe: column 33, lines 12-23, the system recognizes objects and 
then determines whether they belong at a location); and d) deciding whether said 
suspected object is a dangerous object (Monroe: column 33, lines 12-23, detected 
changes signal an alarm). 

Re claim 15, Monroe discloses that the logic processing comprises the steps of: 
a) measuring parameters regarding the pixels in the logic matrix (Monroe: column 33, 
lines 12-23, the system is capable of learning where objects are located by monitoring 
scenes over time); b) comparing said measured parameters to a predetermined table of 
values stored in the memory, whenever said measured parameters equal to one or 
more values in said table, the pixels that relates to said measurement are dangerous 
objects (Monroe: column 33, lines 19-20, any changes activate an alarm). 

Re claim 20, Monroe discloses rotating both imagers of a pair of imagers and 
simultaneously changing their corresponding angle of view (Monroe: Fig. 40, rotation of 
the pole causes the cameras to rotate together). 
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Re claim 23, Monroe discloses a) generating a panoramic image and a map of 
the monitored area by scanning said area, said scanning being performed by rotating at 
least a pair of distinct and identical imagers around their central axis of symmetry 
(Monroe: column 28, lines 62-64, wide angle feature; Fig. 40, two cameras mounted); b) 
obtaining the referenced location of a detected object by observing said object with said 
imagers, said location being represented by the altitude, range and azimuth parameters 
of said object (Monroe: column 28, lines 46-47, GPS receiver provided for generating 
location information; column 28, line 64-column 29, line 2, range finder permits locating 
objects in a precise manner); and c) displaying the altitude value of said object on said 
panoramic image and displaying the range and the azimuth of said object on said map 
(Monroe: column 16, lines 14-15; column 23, lines 32-35, a portable device may display 
maps along with alphanumeric text). 

Re claim 24, Monroe discloses that the imagers are selected from the group 
consisting of: optical imagers, thermal imagers, CCD cameras, CMOS based cameras, 
and Forward Looking Infra Red (FLIR) cameras (Monroe: column 6, lines 5-8). 

Re claim 26, Monroe discloses that the imagers are not identical and do not 
share common central axis of symmetry or of optical magnification but have at least an 
overlapping part of their field of view (Monroe: Fig. 40, infrared sensor 562 and camera 
558). 

Re claim 27, Monroe discloses documenting the activities of the wildlife and 
other dangerous objects, for preventing and reducing from said wildlife and said other 
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dangerous objects to appear at the monitored area (Monroe: column 4, lines 6-8, 
response to an alarm event may include the dispatch of response personnel). 

Claim 28 recites the corresponding apparatus for implementing the method of 
claim 1, and therefore claim 28 has been analyzed and rejected with respect to claim 1 
above. 

Re claim 29, Monroe discloses that the imagers are photographic devices 
selected from the group consisting of: CCD or CMOS cameras, infrared cameras, and 
Forward Looking Infra Red (FLIR) cameras (Monroe: column 6, lines 5-8). 

Re claim 31 , Monroe discloses that both imagers of a pair of imagers are 
identical (Monroe: Fig. 40; column 27, lines 11-24). 

Re claim 32, Monroe discloses that both imagers of a pair of imagers are 
provided with a different lens (Monroe: Fig. 40, separate cameras). 

Re claim 33, Monroe discloses a) elaborator means for obtaining the referenced 
location of a detected object in said controlled space, said location being represented by 
the altitude, range and azimuth parameters of said object (Monroe: column 28, lines 46- 
47, GPS receiver provided for generating location information; column 28, line 64- 
column 29, line 2, range finder permits locating objects in a precise manner); b) means 
for generating a panoramic image and a map of the monitored area (Monroe: column 
28, lines 62-64, wide angle feature); c) means for displaying the altitude value of said 
object on said panoramic image and means for displaying the range and the azimuth of 
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said object on said map (Monroe: column 16, lines 14-15; column 23, lines 32-35, a 
portable device may display maps along with alphanumeric text). 

Re claim 35, Monroe discloses that the elaborator means comprise one or more 
dedicated algorithm installed within a computerized system (Monroe: column 31, lines 
37-56, instructions stored). 

Re claim 36, Monroe discloses a laser range finder being electrically connected 
to a computerized system for measuring the distance of a detected object from said 
laser range finder, said laser range finder transfers to said computerized system data 
representing the distance from a detected object, thereby aiding said computerized 
system to obtain the location of said detected object (Monroe: column 28, line 64- 
column 29, line 2, range finder permits locating objects in a precise manner). 

Re claim 37, Monroe discloses procuring, adjourning and storing in a memory 
files representing a background space (Monroe: column 33, lines 12-23, the system is 
capable of learning where objects are located by monitoring scenes over time). 

Re claim 38, Monroe discloses a set of motors for displacing the imagers, for 
changing the controlled space or sections thereof, and for thereby generating the real- 
time photographic data (Monroe: Fig. 40 and column 27, line 66-column 27, line 17). 

3. Claims 2, 3, 5-7, 14, and 16 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Monroe (US 6970183 B1) in view of Hsu (US 6741744 B1) in 
view of Lipton (US 6954498 B1). 
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Re claim 2, Monroe discloses a) changing the sections of said stereoscopic 
observation so as to monitor the path of any detected dangerous objects (Monroe: 
column 28, lines 52-59, tracking); b) receiving and storing the data defining the positions 
and the foreseen future path of all authorized bodies (Monroe: column 28, lines 8-15); 
and d) comparatively processing said assumed future path with the foreseen future path 
of all authorized bodies, to determine the possible danger of collision or intrusion 
(Monroe: column 34, lines 56-64, deviations from a pattern trigger an alarm). 

Monroe does not specifically disclose c) extrapolating the data obtained by 
monitoring the path of any detected dangerous objects to determine an assumed future 
path of said objects. However, Lipton discloses an interactive video manipulation 
system, wherein positions are predicted from a tracking algorithm (Lipton: column 7, 
lines 27-37). Since Monroe and Lipton both relate to surveillance, one of ordinary skill 
in the art at the time of the invention would have found it obvious to combine the 
prediction of Lipton with the system of Monroe in order to facilitate the tracking of 
multiple objects in a frame (Lipton: column 7, lines 14-26). 

Re claim 3, Monroe discloses determining an action on dangerous objects that 
will eliminate the danger of collision, intrusion or damage (Monroe: column 4, lines 43- 
57, response to an alarm event). 

Re claim 5, Monroe does not disclose that the action is change in their assumed 
future path the dangerous object. However, Lipton discloses that the predicted position 
is compared to the actual position to determine a more accurate predicted path (Lipton: 
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column 7, lines 27-37). Since Monroe and Lipton both relate to surveillance, one of 
ordinary skill in the art at the time of the invention would have found it obvious to 
combine the prediction of Lipton with the system of Monroe in order to facilitate the 
tracking of multiple objects in a frame (Lipton: column 7, lines 14-26). 

Re claim 6, Monroe discloses determining an action on an authorized body that 
will eliminate the danger of collision, intrusion or damage (Monroe: column 4, lines 6-8, 
response to an alarm event may include the dispatch of response personnel). 

Re claim 7, neither Monroe nor Lipton specifically discloses that the action is a 
delay in the landing or take-off of an aircraft or a change of the landing or take-off path 
of said aircraft. However, the Examiner takes Official Notice that one of ordinary skill in 
the art at the time of the invention would have found it obvious that in the event that a 
hazardous item endangered an aircraft's take-off or landing, appropriate diversionary 
measures would be taken to eliminate possible threats to the aircraft, passengers, 
and/or cargo. 

Re claim 14, Monroe does not specifically disclose the claimed features. 
However, Lipton discloses a) generating an average image from the current one or 
more images (Monroe: column 33, lines 12-23, the system learns that an object belongs 
or does not belong at a specific location and recognizes whether the objects in a frame 
belong there); b) generating a derivative matrix from said average image for 
emphasizing relatively small objects at each image from said one or more images, 
which might be potential dangerous objects (Lipton: column 5, lines 41-60); c) storing 
said derivative matrix in the memory as part of an image database, and comparing said 
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derived matrix with a previous derivative matrix stored in said memory as part of said 
image database, said previous derivative matrix being derived from one or more images 
that was taken from the exact imager angle as of said average image (Lipton: column 5, 
lines 41-60); d) from the comparison, generating an error image, wherein each pixel in 
said error image represents the error value between said derivative matrix and said 
previous derivative matrix (Lipton: column 5, lines 41-60); e) comparing the value of 
each pixel from said error image to a threshold level, said threshold level being 
dynamically determined to each pixel in the error image statistically according the 
previous pixel values stored in the memory as a part of a statistic database (Lipton: 
column 5, line 61 -column 6, line 18); f) whenever a pixel value in said error image 
exceeds said threshold level, generating a logic matrix in which the location of said pixel 
value is set to a predetermined value (Lipton: column 5, line 61 -column 6, line 18); and 
g) upon completing comparing each error value to said threshold level, for the entire 
current images, transferring said generated logic matrix to the logic process stage 
(Lipton: column 6, lines 41-44, pixels are clustered into regions of interest). Since 
Monroe and Lipton both relate to surveillance, one of ordinary skill in the art at the time 
of the invention would have found it obvious to combine the prediction of Lipton with the 
system of Monroe in order to facilitate the tracking of multiple objects in a frame (Lipton: 
column 7, lines 14-26). 

Re claim 16, Monroe does not specifically disclose the claimed features. 
However, Lipton discloses that the parameters are selected from the group consisting of 
the dimension of an adjacent group of pixels, the track that one or more adjacent pixels 
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created in the logic matrix, direction, speed, size and location of an object that is 
created from a group of pixels (Lipton: column 7, lines 27-37). Since Monroe and Lipton 
both relate to surveillance, one of ordinary skill in the art at the time of the invention 
would have found it obvious to combine the prediction of Lipton with the system of 
Monroe in order to facilitate the tracking of multiple objects in a frame (Lipton: column 7, 
lines 14-26). 

4. Claims 18, 21, 22, 25, 30, and 34 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Monroe (US 6970183 B1) in view of Hsu (US 6741744 B1) 
in view of Milgram et al. (US 5175616 A, hereinafter referred to as "Milgram"). 

Re claim 18, Monroe does not specifically disclose the claimed features. 
However, Milgram discloses that the camera separation affects the focal point of the 
imaging system, wherein by adjusting the camera configuration to meet the needs of the 
operator, the task of the operator is theoretically facilitated (Milgram: Figs. 1 1 A-1 1 E and 
column 28, line 35 through column 29, line 3). Therefore, one of ordinary skill in the art 
at the time of the invention would have found it obvious that a distance of 0.5 to 50 
meters between cameras would simply be categorized as a design choice by the 
operator for facilitating his needs. Since Monroe and Milgram relate to surveillance 
systems, one of ordinary skill in the art at the time of the invention would have found it 
obvious to combine the stereoscopic coordinate system of Milgram with the system of 
Monroe in order to provide enhanced 3-D interaction from the operator (Milgram: 
column 3, lines 41-60 and column 4, lines 10-29). 



Application/Control Number: 10/521,207 Page 13 

Art Unit: 2621 

Re claim 21, Monroe discloses providing at least one encoder and at least one 
reset sensor for determining the angle of each imager, said encoder and reset sensor 
being provided to each axis that rotates an imager (Monroe: Fig. 55 and column 37, 
lines 44-49, encoder 1044; column 4, lines 36-40, the system may be configured and 
reconfigured). 

Re claim 22, Monroe discloses that the reset sensor provides the initiation angle 
of the imager at the beginning of the scanning of a sector and the encoder provides the 
current angle of the imager during the scanning of the sector (Monroe: column 4, lines 
36-40, the system may be configured and reconfigured). 

Claim 25 has been analyzed and rejected with respect to claim 18 above. 

Claim 30 has been analyzed and rejected with respect to claim 18 above. 

Re claim 34, Monroe does not specifically disclose that the means for displaying 
the monitored area comprise three-dimensional software graphics for indicating the 
location of each detected object as a three-dimensional image. However, Milgram 
discloses a stereoscopic video-graphic coordinate specification system, wherein 
stereoscopic video is employed in remote surveillance systems to generate 3-D 
information (Milgram: column 1, lines 5-10 and column 3, line 63-column 4, line 2). 
Since Monroe and Milgram relate to surveillance systems, one of ordinary skill in the art 
at the time of the invention would have found it obvious to combine the stereoscopic 
coordinate system of Milgram with the system of Monroe in order to provide enhanced 



Application/Control Number: 10/521,207 
Art Unit: 2621 

3-D interaction from the operator (Milgram: 
10-29). 
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column 3, lines 41-60 and column 4, lines 



5. Claims 12 and 13 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Monroe (US 6970183 B1) in view of Hsu (US 6741744 B1) in view of Lipton 
(US 6954498 B1) in view of Milgram et al. (US 5175616 A, hereinafter referred to as 
"Milgram"). 

Re claim 12, Monroe does not specifically disclose that the pixel processing 
comprises the step of: a) mathematically processing each pixel in a current photo for 
detecting suspected objects. However, Lipton discloses performing a sum of absolute 
differences (SAD) (Lipton: column 7, lines 38-51, sum of absolute differences (SAD) 
performed), wherein each pixel in a block is compared to the corresponding pixel in 
another block, and the differences are summed. Since Monroe and Lipton both relate to 
surveillance, one of ordinary skill in the art at the time of the invention would have found 
it obvious to combine the prediction of Lipton with the system of Monroe in order to 
facilitate the tracking of multiple objects in a frame (Lipton: column 7, lines 14-26). 

Neither Monroe nor Lipton specifically discloses that b) whenever a suspected 
object is detected, providing images at the same time period and of the same monitored 
section by means of both imagers of the one or more pairs of imagers, for generating 
three dimensional data related to said suspected object. However, Milgram discloses a 
stereoscopic video-graphic coordinate specification system, wherein stereoscopic video 
is employed in remote surveillance systems to generate 3-D information (Milgram: 
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column 1, lines 5-10 and column 3, line 63-column 4, line 2). Since Monroe, Lipton, and 
Milgram relate to surveillance systems, one of ordinary skill in the art at the time of the 
invention would have found it obvious to combine the stereoscopic coordinate system of 
Milgram with the system of Monroe and Lipton in order to provide enhanced 3-D 
interaction from the operator (Milgram: column 3, lines 41-60 and column 4, lines 10- 
29). 

Re claim 13, Monroe discloses that whenever the pixel processing detects a 
moving object, further comprising the steps of: a) comparing the current images to an 
average image generated from the previous stored images, said previous stored images 
and said current image being captured at the same imager angle (Monroe: column 33, 
lines 12-23, the system learns that an object belongs or does not belong at a specific 
location and recognizes whether the objects in a frame belong there). 

Monroe does not specifically disclose b) generating a comparison image from the 
difference in the pixels between said average image said current image, each pixel in 
said comparison image representing an error value; c) comparing each error value to a 
threshold level, said threshold level being dynamically determined to each pixel in the 
image matrix statistically according the previous pixel values stored in the memory as a 
statistic database; d) whenever a pixel value in said comparison image exceeds said 
threshold level, generating a logic matrix in which the location of said pixel value is set 
to a predetermined value; and e) upon completing comparing each error value to said 
threshold level, for the entire current images, transferring said generated logic matrix to 
the logic process stage. However, Lipton discloses b) generating a comparison photo 
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from the difference in the pixels between said average photo said current photo, each 
pixel in said comparison photo represents an error value (Lipton: column 5, lines 41-60); 
c) comparing each error value to a threshold level, said threshold level is dynamically 
determined to each pixel in the photo matrix statistically according the previous pixel 
values stored in the memory as a statistic database (Lipton: column 5, lines 41-60); d) 
whenever a pixel value in said comparison photo exceeds said threshold level, 
generating a logic matrix in which the location of said pixel value is set to a 
predetermined value (Lipton: column 5, line 61 -column 6, line 18); and e) upon 
completing comparing each error value to said threshold level, for the entire current 
photos, transferring said generated logic matrix to the logic process stage (Lipton: 
column 6, lines 41-44, pixels are clustered into regions of interest). Since Monroe and 
Lipton both relate to surveillance, one of ordinary skill in the art at the time of the 
invention would have found it obvious to combine the prediction of Lipton with the 
system of Monroe in order to facilitate the tracking of multiple objects in a frame (Lipton: 
column 7, lines 14-26). 

6. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Monroe (US 6970183 B1) in view of Hsu (US 6741744 B1) in view of Goldenberg et 
al. (US 6113343 A, hereinafter referred to as "Goldenberg"). 

Re claim 4, neither Monroe nor Lipton discloses that the action is the destruction 
of the dangerous object. However, Goldenberg discloses an explosives disposal robot, 
which uses surveillance techniques to locate and approach, ultimately destroying, 
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hazardous items (Goldenberg: Abstract section). Since Monroe, Lipton, and 
Goldenberg use surveillance techniques, one of ordinary skill in the art at the time of the 
invention would have found it obvious to combine the destructive property of 
Goldenberg with the system of Monroe and Lipton in order to safely assist in explosive 
ordinance disposal without endangering humans. 



Conclusion 

7. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 



Contact 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to CHRISTOPHER FINDLEY whose telephone number is 
(571 )270-1 1 99. The examiner can normally be reached on Monday-Friday (8:30 AM- 
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